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DETAILED ACTION 

1 . This action is responsive to communication: election filed 8/18/05 to the 
application filed on 05/30/97. 

2. Applicant elected Group I (claims 122-173) without traverse. 

3. Claims 122-178 are pending in the case. Claims 122, 157, 166 are independent 
claims. 

Claim Objections 

4. Claim 122 is objected to as including a typographical error: the word "object" 
should be in plural within "before all data slices of the first and second multimedia 
obiecf dine 13). 

Claim Rejections - 35 USC § 101 

5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

6. Claims 166-173 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non-statutory subject matter. Claims 166-173 are directed to a computer 
implemented multimedia document, which is merely a non-functional descriptive 
material since they do not recite at least one functional element. See MPEP 2106. 



7. 



Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 
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The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

8. Claims 124 and 130 are rejected under 35 U.S.C. 112, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

Regarding claim 124, which is dependent on claim 122, it is impossible that all of the 
data slices of the third multimedia object are progressively rendered before any of the 
data slices of the first and second multimedia objects are progressively rendered. 
As recited in claim 122, the data slices of the first and second multimedia objects are 
progressively rendered before all data slices of the first and second multimedia objects 
are received , and the data slices of the third multimedia object are progressively 
rendered only after a sufficient amount of the data slices of the third multimedia object 
are received to enable rendering the third object. 

Thus, based on claim 122, it appears that the third multimedia object can not be 
rendered before the rendering of the first and second multimedia object. 

Regarding claim 130, which is dependent on claim 122, it is impossible that rendering of 
the first, second, and third files is delayed until all of the objects of the third file are 
received by the recipient. 
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As recited in claim 122, the data slices of the first and second multimedia objects are 
progressively rendered before all data slices of the first and second multimedia objects 
are received , and the data slices of the third multimedia object are progressively 
rendered only after a sufficient amount of the data slices of the third multimedia object 
are received to enable rendering the third object. 

Thus, based on claim 122, it appears that the first and the second multimedia objects 
are rendered before the rendering of the third multimedia object, and thus such delay 
does not happen. 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 122-123, 125-129, 132, 134-136, 140, 143-146, 153-173 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Near et al. (US Pat No. 5,995,091 , 
11/30/99, filed 5/10/96). 

Regarding independent claim 122, Near discloses: 

- receiving author specification of multimedia content and choreography 

information that indicates a temporal order for rendering the multimedia content, 
the multimedia content being defined by at least first, second, and third 
multimedia objects (figure 2, col 3, line 28 to col 4, line 15, col 9, lines 51-67: 
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the arrangement of the interleaved playback includes media portions such as 
video chunk 1 , audio chunk 1 , video chunk 2, video chunk 3 where each portion 
has associated timestamp, engine destination, and message shows the 
choreography information associated with media portion and the temporal order 
of display among the objects; the authoring tool used by an author for specifying 
images and sounds for playback at specified times implies the author 
specification of multimedia content and choreography information for rendering 
the multimedia content defined by at least first, second, and third multimedia 
objects) 

- interleaving data slides of the first and second multimedia objects with each other 
and adding the interleaved data slides to a multimedia document (col 9, lines 20- 
67) 

- adding the data slices of the first and second multimedia objects to the 
multimedia document without interleaving the data slices of the third multimedia 
object with data slices of other objects in the multimedia document (col 9, lines 
21-67, col 13, lines 1-34: interleaving multimedia objects and adding these 
objects to the multimedia document; some separate audio and video are 
combined to reproduce the playback output shows that said separate audio and 
video are added to the multimedia document playback without interleaving) 

- streaming the multimedia document to a recipient for rendering according to the 
choreography information (col 3, line 64 to col 4, line 61, col 7, line 37 to col 8, 
line 20, col 12, line 52 to col 13, line 34) 
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Near does not explicitly disclose said streaming is carried out such that the data slices 
of the first and second multimedia objects are progressively rendered before all data 
slices of the first and second multimedia objects are received and the data slices of the 
third multimedia object are progressively rendered only after a sufficient amount of the 
data slices of the third multimedia object are received to enable rendering of the third 
multimedia object. 

However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have combined the feature of progressively rendering the first 
and second multimedia objects before all of their data slices are received, and 
progressively rendering the third multimedia object only after a sufficient amount of its 
data slices is received for the following reason. The interleaving of multimedia data was 
known to help speeding up the rendering process. Therefore, the first and second 
multimedia objects, which are interleaved, are rendered before all of their data are 
received. The third object, which is not interleaved, is rendered in a normal process, 
which means it is rendered when receiving all of its data. 

Regarding claim 123, which is dependent on claim 122, Near discloses that the data 
slices of the first, second, and third multimedia objects are rendered progressively 
together (col 3, line 28 to col 4, line 15, col 9, lines 51-67: playback the multimedia 
object shows that data of a plurality of multimedia objects including the first, second, 
and third objects are progressively rendered). 
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Regarding claim 125, which is dependent on claim 122, Near discloses that all of the 
data slices of the third multimedia object are progressively rendered after any of the 
data slices of the first and second multimedia objects are progressively rendered (see 
claim 122). 

Regarding claim 126, which are dependent on claim 122, Near discloses that the data 
slices of the first and second multimedia objects can be progressively rendered 
independent of bandwidth without degradation (col 2, lines 61-67: generating a 
bandwidth-controlled presentation data stream, thereby controlling network load and 
providing predictable performance on a variety of playback systems implies that the 
playback performance is not dependent on the bandwidth since the network load while 
sending media data is controlled via the authoring tool). 

Regarding claim 127, which is dependent on claim 122, Near discloses that the data 
slices of the third multimedia object cannot be progressively rendered independent of 
bandwidth without degradation (col 13, lines 1-34: since some specified audio and 
image files are separate from the other multimedia objects, so they are not interleaved 
with the other multimedia objects, rendering of these objects is performed when all of 
the data slices of these objects are received, and thus in a slower process). 

Regarding claim 128, which is dependent on claim 122, Near discloses that the data 
slices of the third multimedia object are rendered only after all of the data slices of the 
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third multimedia object are received (as in claim 122, rendering of the third object 
occurs when all data slices of the third object are received). 

Regarding claim 129, which is dependent on claim 122, Near discloses that the 
choreography information comprises an indication of the author's intent that the first, 
second and third files be rendered substantially together (col 3, line 32 to col 4, line 27). 

Regarding claims 132 and 134, which are dependent on claim 122, Near discloses that 
the first object comprises an image file (figure 2, the optional data in the video chunk 
can be image file). 

Regarding claim 135, which is dependent on claim 122, Near discloses that the third 
object comprises a sound file (col 3, line 56 to col 4, line 26). 

Regarding claim 136, which is dependent on claim 122, Near discloses that the third 
object comprises a video file (figure 2). 

Regarding claim 140, which is dependent on claim 122, Near discloses rendering the 
multimedia document in a window on a computer display at the recipient (col 13, lines 
23-34). 
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Regarding claim 143, which is dependent on claim 122, Near discloses that each object 
has an address indicating a player that plays the object (col 8, lines 1-13; col 4, lines 47- 
61 identifying images and sounds to be reproduced during the playback implies 
identifying the address of the media element and the software that plays the media). 

Regarding claim 144, which is dependent on claim 122, Near discloses compressing 
information in each object (col 8, lines 13-29, col 9, lines 20-39; col 4, lines 47-61). 

Regarding claim 145, which is dependent on claim 122, Near discloses creating an 
object in the file (col 9, lines 20-50) and locating player data within an object defining a 
player that plays the object (col 10, lines 40-67: playback media software used to 
identify individual portion of the playback data stream and the manager software calls a 
function for a player element of media data). 

Near does not disclose the created object is an unknown object. However, it would 
have been obvious to one of ordinary skill in the art at the time of the invention was 
made to have modified Near to include the created object being an unknown object for 
the following reason. Near defines the multimedia data for the playback data stream 
where each multimedia element is considered as an object in the data stream (figure 2, 
col 51-67). Thus no matter the object is known or unknown, the system always locates 
the player associated with the multimedia element. 
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Regarding claim 146, which is dependent on claim 122, Near discloses two or more of 
the objects have at least one common attribute, including at least one of a command for 
perception of the objects, an ability to pass and receive message, and an ability to 
supply and retrieve the data embodied in the objects (col 4, lines 47-61 : the fact that 
each multimedia object is identified by the interleaver element implies that the 
multimedia objects have a command for perception the object, which is a common 
attribute of the objects) . 

Regarding claim 153, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient input (col 3, line 64 to col 4, line 27, col 7, 
lines 50-64: the fact that the playback of the images and sound data stream with 
specified times is generated by an author for a selected bandwidth shows that the 
temporal order is defined by the author, and thus is maintained independent of a 
recipient). 

Regarding claim 154, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient hardware configuration (col 3, lines 48-55: 
since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient hardware configuration). 

Regarding claim 155, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient software configuration (col 3, lines 48-55: 
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since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient software configuration). 

Regarding claim 156, which is dependent on claim 122, Near discloses that the media 
content is rendered independent of an author-specified bandwidth to be used to send 
the multimedia document (col 8, lines 1-28: no matter the bandwidth selected by the 
author used to send the multimedia document is sufficient or insufficient, rendering of 
the multimedia document is still performed via the playback). 

Claims 157-164 are for a computer device to perform the method claim 122-128, 137, 
and are rejected under the same rationale. 

Claim 165 is for a computer device to perform the method claim 156, and is rejected 
under the same rationale. 

Claims 166-172 are for the document included in method claim 122-128, and are 
rejected under the same rationale. 

Claim 173 is for the document included in method claim 156, and is rejected under the 
same rationale. 
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11. Claims 131 and 133 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Near as applied to claim 122 above, and further in view of Cave et al. (US Pat No. 
5,943,046, 8/24/99, filed 7/19/95). 

Regarding claims 131 and 133, which are dependent on claim 122, Near discloses a 
multimedia system for performance playback multimedia data (col 3, lines 28-47). Near 
does not disclose explicitly that the first and second multimedia objects comprises a text 
file. 

Cave discloses that the multimedia objects include voice, music, pictures and text (col 
2, lines 60-67). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Cave into Near for incorporate text files into the 
multimedia objects for having different types of objects in the multimedia documents. 

12. Claims 141 , 147-148 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Near as applied to claim 140 above, and further in view of Shaw et al., Microsoft 
Office 6-in-1, Que Corporation 1994, pages 379-380, 384-389, 396-402, 419-425, 492- 
496. 

Regarding claim 141, which is dependent on claim 140, Near does not disclose: 
- creating an exclusionary area within the window 
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- locating an object within the exclusionary area, the object being selected from a 
group of objects including a framed image, a slide show, framed text, sound data, 
a separator, or a hyperlink 
Shaw discloses: 

- creating an exclusionary area within the window (page 401 , figure 4.4) 

- locating an object within the exclusionary area, the object being selected from a group 
of objects including a framed image, a slide show, framed text, sound data, a separator, 
or a hyperlink (page 401 , figure 4.4: the data in the area within the window can be text 
and graphics). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Near since Shaw teaches the window features 
for locating objects selected from a group of specific objects providing the advantage to 
include in Near to enhance the use of objects from internet instead of merely video or 
audio from the media database. 

Regarding claims 147 and 148, which are dependent on claims 122 and 147 
respectively, Near does not disclose the document forms a code segment that receives 
image information, and wherein the image information is used to construct an image 
frame for a framed image that is part of the multimedia document. 
Shaw discloses that the document forms a code segment that receives image 
information, and wherein the image information is used to construct an image frame for 
a framed image that is part of the multimedia document (pages 400-401). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Near since Shaw a code segment to control the 
receiving of image information as well as the structure of the media stream providing the 
advantage of utilizing such control to edit the media stream in Near as desired. 

13. Claim 142 is rejected under 35 U.S.C. 103(a) as being unpatentable over Near 
as applied to claim 122 above, and further in view of Johnson (US Pat No. 5,892,847, 
4/6/99, filed 4/22/96). 

Regarding claim 142, which are dependent on claim 122, Near does not disclose 
defining as well as locating the update splash image within the data file. 
Johnson discloses: 

- splash image data defining a splash image and locating the splash image data within 
the data file for displaying the splash image on the computer display (col 4, lines 30-50) 

- further updating the splash image to be displayed (col 4, lines 30-63) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Johnson into Near since Johnson teaches the process of 
displaying of a splash image, which is an element of a multimedia document providing 
the advantage of including a splash image into the image file to enhance the image 
characteristics in Near. 
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14. Claims 149-152 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Near as applied to claim 122 above, and further in view of Caire et al. (US Pat No. 
5,663,962, 9/2/97, filed 9/15/95). 

Regarding claim 149, which is dependent on claim 122, Near discloses a header, 
timestamps, and associated data for a data stream (col 5, line 64 to col 6, line 16). 
Near does not disclose that the choreography information further comprises an object 
archive for storing information about one or more objects, the object archive including 
information about the relationship of the object file with the document, and a multiplex 
section including data for the objects in the document. 
Caire discloses: 

- a header (col 1 , lines 65 to col 2, lines 1-2, each packet in the overall stream 
includes a header) 

- an object archive for storing information about the plurality of object files, the 
object archive including information about the level of each object file with the 
hierarchy (col 1, lines 65 to col 2, lines 1-2, each packet of the multimedia stream 
stores information; col 1 , lines 37-52, it is desired for instance to insert into the 
complete stream also some subtitles to be displayed during the presentation....) 

- a multiplex section including data for each of the object files of the document (col 
1, lines 65 to col 2, lines 1-9, 45-59) 

- the object files in the multiplex section are each played by a player as the 
multiplex object file is received by a receiver (col 1 , lines 65 to col 2, lines 1-2) 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
into Near would provide more detailed options in the relationship of the objects in a 
multimedia document to effectively controlling and changing the presentation of the 
objects. 

Regarding claims 150 and 151, which are dependent on claims 149 and 150 
respectively, Near does not disclose an object number counter indicating the number of 
objects, a plurality of object descriptions, each object description describing a 
corresponding one of the objects, and a choreography group providing information 
about a first group of objects, a group object counter indicating the number objects in 
the choreography group, size and type data for each object, header data, data slices of 
the objects interleaved together, and placing one or more slice size data blocks before 
one or more of the interleaved data slices, each slice size data block corresponding to a 
data slice and providing a size of the corresponding data slice. 
Caire discloses: 

- an object number counter indicating the number of object files (col 2, lines 10-20) 

- a plurality of object descriptions, each object description describing a 
corresponding one of the object files (col 1 , lines 65 to col 2, lines 1-2, the 
header includes information of the type of a packet in the multimedia stream) 
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- a choreography group providing information about a first group of object files (col 
1 , lines 65 to col 2, lines 1-2, packets of different types are included in the overall 
stream as a sequence of intervals wherein the type of a packet is disclosed in the 
heading are considered as a choreography group providing information about the 
object files) 

- size and type data for each object file (col 1 , lines 65 to col 2, lines 1-2, data type 
of each packet in the multimedia stream) 

- header data (col 1 , lines 65 to col 2, lines 1-2, each packet includes a header) 

- the data slices of the object files interleaved together (col 1 , lines 65 to col 2, 
lines 1-2, the overall stream is structured as a sequence of intervals called 
packets, each of which contains data of single type, indicated in a header of the 
packet itself; since data of different types are arranged in the sequence of 
intervals called packets, the packets which are equivalent to the object files, are 
interleaved together) 

- a first player pointer including an address of a player that plays the choreography 
group (col 2, lines 3-9, for each interval, the multiplexer has to decide from which 
the input stream it should take the data in order to construct the packets; this 
implies that the multiplexer has to decide where to point to play the overall 
stream which is equivalent to the choreography group as mentioned above) 

- locating a plurality of slice size data blocks before the interleaved data slices, 
each slice size data block corresponding to one of the data slices and providing a 
size of the corresponding data slice (col 4, lines 45-53, the number of data bytes 
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and the number of header bytes in each packet show the size of each packet 

which is equivalent to the data block) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 

Regarding claim 152, which is dependent on claim 150, Near does not disclose a non- 
multiplex section following the multiplex section where the non-multiplex section 
includes one or more separate objects that are not played by a player as the separate 
object files are received by a receiver. 

Caire discloses a plurality of separate object files that are not played by a player as the 
separate object files are received by a receiver (col 1 , lines 37-45, ...video and audio 
information have to be separated again, by an inverse of demultiplexing process, as 
presentation occurs on different devices...). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 
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Response to Arguments 

15. Applicant's arguments filed 5/10/05 have been fully considered but they are not 
persuasive. 

Applicants argue that Near does not disclose "adding data slices of the third multimedia 
object to the multimedia object without interleaving the data slices of the third 
multimedia object with data slices of other objects in the multimedia document" 
(Remarks, page 12). 
Examiner respectfully disagrees. 

First of all, it is not clear what Applicants want to argue since the claim recites adding 
data slices of the third multimedia object to the multimedia document whereas the 
argument refers to adding data slices of the third multimedia object to the multimedia 
object . 

Near discloses the argued feature. See claim rejection. 

Allowable Subject Matter 

16. Claims 137-139 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

Conclusion 

17. THIS ACTION IS MADE FINAL Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

18. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Yager et al. (US Pat No. 5,983,236, 1 1/9/99, filed 7/28/97, priority 7/20/94). 

Fukushima et al. (US Pat No. 5,798,995, 8/25/98, filed 2/14/97). 

Tarr et al. (US Pat No. 5,935,004, 8/10/99, filed 8/28/95). 

Baumgartner et al. (US Pat No. 5,642,171 , 6/24/97, filed 6/8/94). 
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19. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 571-272- 
4125. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-4125. 
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For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
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